{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "龙曲良老师课程笔记/05_Pytorch进阶教程.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyOIq86ZSv91YzJ98CaIg6q4",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jinxiqinghuan/learn_pytorch/blob/main/%E9%BE%99%E6%9B%B2%E8%89%AF%E8%80%81%E5%B8%88%E8%AF%BE%E7%A8%8B%E7%AC%94%E8%AE%B0/05_Pytorch%E8%BF%9B%E9%98%B6%E6%95%99%E7%A8%8B.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c9ulWJJrysc-"
      },
      "source": [
        "# Pytorch进阶教程"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mpBYdCiaywVP"
      },
      "source": [
        "import torch"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vNbydX9Oy0uM"
      },
      "source": [
        "## 1.Broadcasting\r\n",
        "\r\n",
        "**key idea:**\r\n",
        "* insert i dim ahead\r\n",
        "* expand dims with size 1 to same size\r\n",
        "* feature maps: [4, 32, 14, 14]\r\n",
        "* bias: [32, 1, 1]=> [1, 32, 1, 1] => [4, 32, 14, 14]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RVnKEs5hpRhV"
      },
      "source": [
        "龙老师只有原理讲解，没有代码，一下代码来自：https://pytorch.org/docs/stable/notes/broadcasting.html"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ciemwd_YpgEK"
      },
      "source": [
        "Two tensors are “broadcastable” if the following rules hold:\r\n",
        "\r\n",
        "\r\n",
        "*   Each tensor has at least one dimension.\r\n",
        "\r\n",
        "*   When iterating over the dimension sizes, starting at the trailing dimension, the dimension sizes must either be equal, one of them is 1, or one of them does not exist.\r\n",
        "\r\n",
        "\r\n",
        "\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ml3S3Q9Jqbst"
      },
      "source": [
        "**For Example:**\r\n",
        "\r\n",
        "1. same shapes are always broadcastable (i.e. the above rules always hold)\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RJ8YUtdrpQd6",
        "outputId": "7fc39762-2110-4431-c763-018ede02c53f"
      },
      "source": [
        "x = torch.empty(5, 7, 3)\r\n",
        "y = torch.empty(5, 7, 3)\r\n",
        "x.shape == y.shape"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Uo3X5kwTqnih"
      },
      "source": [
        "2. x and y are not broadcastable, because x does not have at least 1 dimension\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FUClXKYbyzm8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c96fdc33-0977-4d31-8f81-d52bd28f09ab"
      },
      "source": [
        "x = torch.empty((0,))\r\n",
        "x, x.shape"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([]), torch.Size([0]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D_LwQAdZrSOe",
        "outputId": "a81b3d39-afc1-416e-a441-8934d3d3bd3e"
      },
      "source": [
        "y = torch.empty(2, 2)\r\n",
        "y, y.shape"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([[4.1494e-35, 0.0000e+00],\n",
              "         [1.8578e-01, 3.9177e-02]]), torch.Size([2, 2]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LRLVuPhkrKH_"
      },
      "source": [
        "3. can line up trailing dimensions\r\n",
        "x and y are broadcastable.\r\n",
        "\r\n",
        "  1st trailing dimension: both have size 1\r\n",
        "\r\n",
        "  2nd trailing dimension: y has size 1\r\n",
        "\r\n",
        "  3rd trailing dimension: x size == y size\r\n",
        "\r\n",
        "  4th trailing dimension: y dimension doesn't exist\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9NdWGhL5q4IG",
        "outputId": "8bb97cb4-ca9e-46e9-af1f-1fddbda55e52"
      },
      "source": [
        "x = torch.empty(5, 3, 4, 1)\r\n",
        "y = torch.empty(  3, 1, 1)\r\n",
        "x.shape, y.shape"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([5, 3, 4, 1]), torch.Size([3, 1, 1]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y4dQdPWuuZL0",
        "outputId": "a4aa0617-d1f4-49d7-9157-c61d19f9ecf6"
      },
      "source": [
        "(x + y).shape"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([5, 3, 4, 1])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5uP1lKSXtQC5"
      },
      "source": [
        "4. but:\r\n",
        "\r\n",
        "  x and y are not broadcastable, because in the 3rd trailing dimension 2 != 3"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0GVIBgO_tP06"
      },
      "source": [
        "x = torch.empty(5, 2, 4, 1)\r\n",
        "y = torch.empty(  3, 1, 1)"
      ],
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KDj20PhLuUsk"
      },
      "source": [
        "# x + y"
      ],
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hx_iyikSt48M"
      },
      "source": [
        "If two tensors x, y are “broadcastable”, the resulting tensor size is calculated as follows:\r\n",
        "\r\n",
        "* If the number of dimensions of x and y are not equal, prepend 1 to the dimensions of the tensor with fewer dimensions to make them equal length\r\n",
        "\r\n",
        "* Then, for each dimension size, the resulting dimension size is the max of the sizes of x and y along that dimension."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hev6DkfMuEFo"
      },
      "source": [
        "**For example**\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C-VIjEscuk1g",
        "outputId": "89eacf27-b138-400c-db58-bfbb0e16ec66"
      },
      "source": [
        "x = torch.empty(5, 1, 4, 1)\r\n",
        "y = torch.empty(  3, 1, 1)\r\n",
        "(x + y).shape"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([5, 3, 4, 1])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uNpObgu2y7s8"
      },
      "source": [
        "## 2.合并与分割\r\n",
        "\r\n",
        "* cat\r\n",
        "* stack\r\n",
        "* split(按长度拆分)\r\n",
        "* chunk(按数量拆分)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "22VfCQ_1yCz1"
      },
      "source": [
        "### cat"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "geGXmgqGy_lV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c2866884-ee23-4cd0-f090-307cc78f3481"
      },
      "source": [
        "a = torch.rand(4, 32, 8) # 四个班级32名同学的八门课程的成绩单\r\n",
        "a.shape"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 32, 8])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JvtIM3zcwE6N",
        "outputId": "e06263b2-d6c0-4a8d-b078-12e3e938cf14"
      },
      "source": [
        "b = torch.rand(5, 32, 8)\r\n",
        "b.shape"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([5, 32, 8])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yZ5wvJRfwKze",
        "outputId": "81de8b7c-c9ff-46a9-843e-03773e305938"
      },
      "source": [
        "torch.cat([a, b], dim=0).shape"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([9, 32, 8])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DYnT52--wSnj"
      },
      "source": [
        "a1 = torch.rand(4, 3, 32, 32)\r\n",
        "a2 = torch.rand(5, 3, 32, 32)"
      ],
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_TVZH2eywxXR",
        "outputId": "89a9e7cc-d262-4da8-b8e3-a1262bd09590"
      },
      "source": [
        "torch.cat([a1, a2], dim=0).shape"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([9, 3, 32, 32])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JlNvRTfQw4f1",
        "outputId": "d6ad3552-6a59-4dad-f919-e61cde22dedb"
      },
      "source": [
        "a2 = torch.rand(4, 1, 32, 32)\r\n",
        "a1.shape, a2.shape"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([4, 3, 32, 32]), torch.Size([4, 1, 32, 32]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1KnFkc0bxRP-",
        "outputId": "3b889bfa-8ff8-4f86-f1bd-b08f0315e2a1"
      },
      "source": [
        "torch.cat([a1, a2], dim=1).shape"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 4, 32, 32])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rGs9ZM5GxlX9",
        "outputId": "005c9f9b-d6ce-4980-d963-8094973a9031"
      },
      "source": [
        "a1 = torch.rand(4, 3, 16, 32)\r\n",
        "a2 = torch.rand(4, 3, 16, 32)\r\n",
        "torch.cat([a1, a2], dim=2).shape"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 3, 32, 32])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1lpdainOyFhb"
      },
      "source": [
        "###stack"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f9n7k-XYx4_X",
        "outputId": "80359aa6-9c60-4c0d-d53a-b97beff943af"
      },
      "source": [
        "torch.cat([a1, a2], dim=2).shape"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 3, 32, 32])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "66M16FwSyPZ7",
        "outputId": "ca651d01-e08d-49e6-ec0e-335437e47e2d"
      },
      "source": [
        "torch.stack([a1, a2], dim=2).shape # 创建了一个新的维度"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 3, 2, 16, 32])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c8SW59nVzMTA"
      },
      "source": [
        "假设这样一个场景，两位老师统计了两个班各32名同学的8门课成绩，想把他们合成一个表格，如果用cat,就无法区分这两个班级了，这个时候要使用stack（这个举例不是很合适，因为用stack两个班级的同学数必须相同）"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dZCeXJEpyZ2j",
        "outputId": "ddfeee04-5d10-4c06-db7d-53ba294da473"
      },
      "source": [
        "a = torch.rand(32, 8)\r\n",
        "b = torch.rand(32, 8)\r\n",
        "torch.stack([a, b], dim=0).shape"
      ],
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([2, 32, 8])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fFPVTJLC0I3h"
      },
      "source": [
        "### split"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QX9qKEKqy-Gy",
        "outputId": "ebd97ffc-c5c2-4d06-c87d-760ab5d9a602"
      },
      "source": [
        "b = torch.rand(32, 8)\r\n",
        "a.shape"
      ],
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([32, 8])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3zOcQll70RUG",
        "outputId": "765aa0fa-5357-4a06-bf0b-49945023ee6d"
      },
      "source": [
        "c = torch.stack([a, b], dim=0)\r\n",
        "c.shape"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([2, 32, 8])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Oifj1XVm0XIW",
        "outputId": "da6a2681-db7f-4456-aebc-0b502606d594"
      },
      "source": [
        "aa, bb = c.split([1, 1], dim=0)\r\n",
        "aa.shape, bb.shape"
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([1, 32, 8]), torch.Size([1, 32, 8]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9RnfVj9j0g9A",
        "outputId": "f1236ebf-a82b-4b4f-c763-3e4058a787a7"
      },
      "source": [
        "aa, bb = c.split(1, dim=0) # 拆分成N块，每块长度为1\r\n",
        "aa.shape, bb.shape"
      ],
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([1, 32, 8]), torch.Size([1, 32, 8]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "goZZHo271Ftw"
      },
      "source": [
        "# aa, bb = c.split(2, dim=0) # ValueError: not enough values to unpack(expected 2, got 1)"
      ],
      "execution_count": 59,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7cg8k7nb9JpX"
      },
      "source": [
        "### chunk"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gz-YbFfT1Zgs",
        "outputId": "c89932f1-82d9-45fe-f4de-6bb2f7b6441f"
      },
      "source": [
        "b = torch.rand(32, 8)\r\n",
        "a.shape"
      ],
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([32, 8])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tF7uKuET9qFt",
        "outputId": "bb0a1b7b-6635-4951-ebdb-2ec4755bd071"
      },
      "source": [
        "c = torch.stack([a, b], dim=0)\r\n",
        "c.shape"
      ],
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([2, 32, 8])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "V88SMaJ09xgR",
        "outputId": "5f502499-ace3-46a9-a267-17cb270b8643"
      },
      "source": [
        "aa, bb = c.chunk(2, dim=0)\r\n",
        "aa.shape, bb.shape"
      ],
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([1, 32, 8]), torch.Size([1, 32, 8]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VuSLeezO_OXH"
      },
      "source": [
        "## 3.数学运算"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q4WaRbItB-d1"
      },
      "source": [
        "### 加减乘除"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1ZsDmtHW_Q8U",
        "outputId": "873e8468-fb5a-48f0-f5e8-d0f48bf03357"
      },
      "source": [
        "a = torch.rand(3, 4)\r\n",
        "b = torch.rand(4)\r\n",
        "a, b"
      ],
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([[0.0959, 0.6001, 0.4030, 0.7851],\n",
              "         [0.0390, 0.4698, 0.0845, 0.8910],\n",
              "         [0.8941, 0.9692, 0.9845, 0.0893]]),\n",
              " tensor([0.6872, 0.2317, 0.7131, 0.6929]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zz90qYek_-bZ",
        "outputId": "f0815c2c-c7e3-47f5-b858-70dda6eaab2a"
      },
      "source": [
        "a + b"
      ],
      "execution_count": 68,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.7832, 0.8318, 1.1161, 1.4780],\n",
              "        [0.7263, 0.7015, 0.7976, 1.5840],\n",
              "        [1.5813, 1.2009, 1.6976, 0.7822]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 68
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rjAMSKb_ABNB",
        "outputId": "69271c8c-a75a-43fd-954a-130407d892d6"
      },
      "source": [
        "torch.add(a, b)"
      ],
      "execution_count": 69,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.7832, 0.8318, 1.1161, 1.4780],\n",
              "        [0.7263, 0.7015, 0.7976, 1.5840],\n",
              "        [1.5813, 1.2009, 1.6976, 0.7822]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 69
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4UJNjhAgAL4Y",
        "outputId": "ca5446b5-9126-4811-e948-212be9146e46"
      },
      "source": [
        "torch.all(torch.eq(a-b, torch.sub(a, b)))"
      ],
      "execution_count": 70,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(True)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 70
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0N67_tw2AYPU",
        "outputId": "684ba5ec-3100-4a8d-bc97-6f0ef7ca1501"
      },
      "source": [
        "torch.all(torch.eq(a*b, torch.mul(a, b)))"
      ],
      "execution_count": 72,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(True)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 72
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hHNpBcvBAe-G",
        "outputId": "a7d1b01f-d910-41b2-feb0-44fe58ec7800"
      },
      "source": [
        "torch.all(torch.eq(a/b, torch.div(a, b)))"
      ],
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(True)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 73
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tx5UWU00CHWC"
      },
      "source": [
        "### 矩阵相乘matmul"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X6VUuLtjB1dB",
        "outputId": "06cd6230-ff63-484f-d82e-85d5701e1ac6"
      },
      "source": [
        "b = torch.ones(2, 2)\r\n",
        "b"
      ],
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1., 1.],\n",
              "        [1., 1.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_rQCVYSyCRql",
        "outputId": "1a1dab9d-0d12-448f-c6d3-db583e6c2761"
      },
      "source": [
        "a = b*3\r\n",
        "a"
      ],
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[3., 3.],\n",
              "        [3., 3.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c-1vI70UCcda",
        "outputId": "249d1e5b-cc67-42d7-c2bc-8cb31fcf0048"
      },
      "source": [
        "torch.mm(a, b)"
      ],
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[6., 6.],\n",
              "        [6., 6.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 81
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9q1FdDgaChIh",
        "outputId": "6db9d56e-00a2-44a7-c915-f576d182c247"
      },
      "source": [
        "torch.matmul(a, b)"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[6., 6.],\n",
              "        [6., 6.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 82
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gd2ODNg0CkCC",
        "outputId": "f6c220ee-b888-4a97-b5d2-4bff18876792"
      },
      "source": [
        "a@b"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[6., 6.],\n",
              "        [6., 6.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5puqNdQNCmcV"
      },
      "source": [
        "a = torch.rand(4, 784)\r\n",
        "x = torch.rand(4, 784)\r\n",
        "w = torch.rand(512, 784) # (out, in)"
      ],
      "execution_count": 84,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YVbF3btTDBox",
        "outputId": "c9b73c34-42a0-4515-c240-96d35afd89e9"
      },
      "source": [
        "(x@w.t()).shape #如果是高纬的使用transpose"
      ],
      "execution_count": 90,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 512])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 90
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uyDDvLmiD527"
      },
      "source": [
        "高纬矩阵相乘"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iPXv3a30DC_9"
      },
      "source": [
        "a = torch.rand(4, 3, 28, 64)\r\n",
        "b = torch.rand(4, 3, 64, 32)"
      ],
      "execution_count": 91,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_1tJ_-v5EMNZ",
        "outputId": "72d9488d-f181-45ff-d341-f9a8d2520bc8"
      },
      "source": [
        "# torch.mm(a, b).shape\r\n",
        "torch.matmul(a, b).shape"
      ],
      "execution_count": 94,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 3, 28, 32])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 94
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fc4MXnpLEPgw"
      },
      "source": [
        "b = torch.rand(4, 1, 64, 32)"
      ],
      "execution_count": 96,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5lYjGsz7FVOW",
        "outputId": "8b87c397-b1b0-4dd4-ac59-121db95642f4"
      },
      "source": [
        "torch.matmul(a, b).shape"
      ],
      "execution_count": 97,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 3, 28, 32])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 97
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B03RbWB2Fhst"
      },
      "source": [
        "# b = torch.rand(4, 64, 32)\r\n",
        "# torch.matmul(a, b).shape"
      ],
      "execution_count": 100,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "075cmSRvGAyZ"
      },
      "source": [
        "### power"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2mvbHYwAFuuE",
        "outputId": "cd278fbc-39c7-4427-e3f8-12f6231a5f4a"
      },
      "source": [
        "a = torch.full([2, 3], 3)\r\n",
        "a"
      ],
      "execution_count": 101,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[3, 3, 3],\n",
              "        [3, 3, 3]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 101
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GBXs6f0KGaCW",
        "outputId": "5d078149-2519-4a06-ba00-85e45df1b950"
      },
      "source": [
        "a.pow(2)"
      ],
      "execution_count": 102,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[9, 9, 9],\n",
              "        [9, 9, 9]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 102
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nqrFPI5gGcnZ",
        "outputId": "3d5e311e-4459-4163-941b-331f876cb76d"
      },
      "source": [
        "a**2"
      ],
      "execution_count": 103,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[9, 9, 9],\n",
              "        [9, 9, 9]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 103
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eMLn3sxvGd3Y",
        "outputId": "6202cac3-afb6-4469-d7b3-d9dbe2bb9cab"
      },
      "source": [
        "aa = a**2\r\n",
        "aa"
      ],
      "execution_count": 107,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[9, 9, 9],\n",
              "        [9, 9, 9]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 107
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "JUcaOEcDG4Hq",
        "outputId": "234db380-a56f-491f-b241-75d0f99b539f"
      },
      "source": [
        "aa.type()"
      ],
      "execution_count": 114,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'torch.LongTensor'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 114
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yukH3onoGf5z"
      },
      "source": [
        "# aa.sqrt()"
      ],
      "execution_count": 119,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_D5DDANrGmDm"
      },
      "source": [
        "# aa.rsqrt()"
      ],
      "execution_count": 121,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Shx8mp_jGtvB",
        "outputId": "371b7f20-16d6-4c41-975a-03fb12ffa7a3"
      },
      "source": [
        "aa ** 0.5"
      ],
      "execution_count": 123,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[3., 3., 3.],\n",
              "        [3., 3., 3.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 123
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jgZsb0HGI1AF"
      },
      "source": [
        "### exp and log"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vxYLY64QIYGv",
        "outputId": "b24cdcbd-e804-42ce-93d3-21ba4bf468f9"
      },
      "source": [
        "a = torch.exp(torch.ones(2, 2))\r\n",
        "a"
      ],
      "execution_count": 124,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[2.7183, 2.7183],\n",
              "        [2.7183, 2.7183]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 124
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zzKBkQBuIqsu",
        "outputId": "15f30add-83e4-4e91-e5e5-c8a60f221f70"
      },
      "source": [
        "torch.log(a)"
      ],
      "execution_count": 125,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1., 1.],\n",
              "        [1., 1.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 125
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uIP6qVMqI4av"
      },
      "source": [
        "### 近似值 Approximation\r\n",
        "* floor().ceil()\r\n",
        "* round()\r\n",
        "* trunc().frac()"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s0LJq7QYIxYX",
        "outputId": "7272e7b7-b20d-4471-a28d-f33665fc25c6"
      },
      "source": [
        "a = torch.tensor(3.14)\r\n",
        "a"
      ],
      "execution_count": 126,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(3.1400)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 126
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "auhhXIMpJeff",
        "outputId": "0b46bc67-9c98-4a8d-a234-811a9c4c2928"
      },
      "source": [
        "a.floor(), a.ceil(), a.trunc(), a.frac()"
      ],
      "execution_count": 127,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(3.), tensor(4.), tensor(3.), tensor(0.1400))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 127
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5EBL4t8NJjC_",
        "outputId": "f293918a-8bff-4abe-a0a0-31965caa51c1"
      },
      "source": [
        "a = torch.tensor(3.499)\r\n",
        "a.round()"
      ],
      "execution_count": 128,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(3.)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 128
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LFu4YGY8Jrmg",
        "outputId": "31fc40a6-3825-4b8c-aa88-5d3bdb5e2c41"
      },
      "source": [
        "a = torch.tensor(3.5)\r\n",
        "a.round()"
      ],
      "execution_count": 130,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(4.)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 130
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aCGXUdVjJ6rR"
      },
      "source": [
        "### clamp\r\n",
        "裁剪功能，梯度离散和梯度爆炸\r\n",
        "\r\n",
        "train不稳定时，打印梯度的模\r\n",
        "\r\n",
        "w.grad.norm(2)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gBJv4dzOJv4c",
        "outputId": "c9d43daf-e381-4f9b-a301-91df6fd7c2eb"
      },
      "source": [
        "grad = torch.rand(2, 3)*15\r\n",
        "grad.max()"
      ],
      "execution_count": 138,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(14.3433)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 138
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FJ8tX6oTKf4A",
        "outputId": "6014bb07-dce3-43f5-9dda-5f081ed9b9c1"
      },
      "source": [
        "grad.median()"
      ],
      "execution_count": 139,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(8.4997)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 139
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nKHtx3G-Kogo",
        "outputId": "253f7fc7-c77c-4846-d34d-9e3ae2e33bbe"
      },
      "source": [
        "grad.clamp(10) # 小于10的赋值为10"
      ],
      "execution_count": 140,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[10.0000, 13.0912, 14.3433],\n",
              "        [10.0000, 10.0000, 12.2009]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 140
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-fhnpUqUKqYR",
        "outputId": "1e30ce04-2c09-409c-9dc1-72d2d21bf6bb"
      },
      "source": [
        "grad"
      ],
      "execution_count": 136,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ 7.2440,  6.6741, 11.2324],\n",
              "        [ 9.2214,  0.7991,  4.2448]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 136
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jUe6JMNMKsY0",
        "outputId": "6e893d90-ec40-44d1-d885-4971000e1187"
      },
      "source": [
        "grad.clamp(0, 10) # 0到10之间"
      ],
      "execution_count": 137,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ 7.2440,  6.6741, 10.0000],\n",
              "        [ 9.2214,  0.7991,  4.2448]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 137
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FJz5XBnbQ6mq"
      },
      "source": [
        "## 4.统计属性\r\n",
        "* norm \r\n",
        "* mean sum\r\n",
        "* prod\r\n",
        "* max, min, argmin, argmax(最大最小值的位置)\r\n",
        "* kthvalue, topk"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AYdT67WhSGxm"
      },
      "source": [
        "### norm"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "73gbb7CtT2BK"
      },
      "source": [
        "torch.set_default_tensor_type(torch.DoubleTensor)"
      ],
      "execution_count": 157,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bpwOuvVBKxKk",
        "outputId": "647131c9-e7e3-4922-a28c-8f7527a77fe6"
      },
      "source": [
        "a = torch.full([8], 1).double()\r\n",
        "a"
      ],
      "execution_count": 158,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1., 1., 1., 1., 1., 1., 1., 1.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 158
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PU9Asn0rSOol",
        "outputId": "89f2ebab-7527-40bf-bc2c-2ac317942418"
      },
      "source": [
        "b = a.view(2, 4).double()\r\n",
        "b"
      ],
      "execution_count": 159,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1., 1., 1., 1.],\n",
              "        [1., 1., 1., 1.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 159
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I752AuxBSRbc",
        "outputId": "efcb6272-d2d6-4917-8880-169683ce9ab5"
      },
      "source": [
        "c = a.view(2, 2, 2).double()\r\n",
        "c"
      ],
      "execution_count": 160,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[[1., 1.],\n",
              "         [1., 1.]],\n",
              "\n",
              "        [[1., 1.],\n",
              "         [1., 1.]]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 160
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9ulTNwbSSXGv",
        "outputId": "d715082f-5aac-4dbd-aa84-66f64d24ec0c"
      },
      "source": [
        "a.norm(1), b.norm(1), c.norm(1)"
      ],
      "execution_count": 161,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(8.), tensor(8.), tensor(8.))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 161
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_f37XOw6Sv9U",
        "outputId": "1c920d0f-d8d8-4d51-bb0b-84596b23e79d"
      },
      "source": [
        "a.norm(2), b.norm(2), c.norm(2)"
      ],
      "execution_count": 162,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(2.8284), tensor(2.8284), tensor(2.8284))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 162
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-zlGQjguUN-J",
        "outputId": "a849d600-a4c4-4cd4-a746-55b653d451c8"
      },
      "source": [
        "b.norm(1, dim=1)"
      ],
      "execution_count": 164,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([4., 4.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 164
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eg7_Uj4rUSHt",
        "outputId": "92b84158-dc5a-44aa-a56d-d7317ef07251"
      },
      "source": [
        "b.norm(2, dim=1)"
      ],
      "execution_count": 165,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([2., 2.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 165
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nZqHAvTkUZ3a",
        "outputId": "3bc40e5d-2a39-4f3a-d3cf-c22b23e290c5"
      },
      "source": [
        "c.norm(2,dim=0)"
      ],
      "execution_count": 166,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1.4142, 1.4142],\n",
              "        [1.4142, 1.4142]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 166
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aeuKWk4dUfAo",
        "outputId": "07713cb3-a7f6-4800-b1a5-4d7144cecf1b"
      },
      "source": [
        "c.norm(2,dim=0)"
      ],
      "execution_count": 167,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1.4142, 1.4142],\n",
              "        [1.4142, 1.4142]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 167
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "34HhSNNVUu5d"
      },
      "source": [
        "### mean, sum, min, max, prod"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MhtUBO_4UiAM",
        "outputId": "b9f11d26-d3bc-4759-b57c-ec879469104e"
      },
      "source": [
        "a = torch.arange(8).view(2, 4).float()\r\n",
        "a"
      ],
      "execution_count": 169,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0., 1., 2., 3.],\n",
              "        [4., 5., 6., 7.]], dtype=torch.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 169
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ot70qDORU5qH",
        "outputId": "a8a97a95-b74c-4999-febd-d64a2276df71"
      },
      "source": [
        "a.min(), a.max(), a.mean(), a.prod() # prod()累乘"
      ],
      "execution_count": 171,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(0., dtype=torch.float32),\n",
              " tensor(7., dtype=torch.float32),\n",
              " tensor(3.5000, dtype=torch.float32),\n",
              " tensor(0., dtype=torch.float32))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 171
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Sc8RbRtRVCjM",
        "outputId": "d0f83e23-e248-4bc6-8d07-43e79102b55f"
      },
      "source": [
        "a.sum()"
      ],
      "execution_count": 172,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(28., dtype=torch.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 172
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SPzfzH_gVMRQ",
        "outputId": "0c1def52-df8e-463d-b1f1-3caa85a01560"
      },
      "source": [
        "a.argmax(), a.argmin() # 打平以后的索引"
      ],
      "execution_count": 173,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(7), tensor(0))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 173
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZPLhSnGQVPLZ",
        "outputId": "56a37cd4-e638-4582-9cc6-d866837bad60"
      },
      "source": [
        "a[1,1]"
      ],
      "execution_count": 177,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(5., dtype=torch.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 177
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sAlmfcuxWgkB",
        "outputId": "7aa6504c-b1d4-4b2b-b9ce-ed2dde981df4"
      },
      "source": [
        "a = torch.randn(4, 10)\r\n",
        "a[0]"
      ],
      "execution_count": 179,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([-0.8184,  0.0473,  0.1040,  1.5820, -0.4505,  0.1631,  1.2550,  0.1290,\n",
              "        -1.5484, -0.7438])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 179
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8HBsaP4LWlJx",
        "outputId": "256b8307-7f8e-48c4-c763-8def5639ee12"
      },
      "source": [
        "a.argmax()"
      ],
      "execution_count": 180,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(3)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 180
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3_Ed0AHgWrmq",
        "outputId": "66ee4c28-cd80-4a9b-fb9a-226651602342"
      },
      "source": [
        "a.argmax(dim=1) # 每一行的最大值"
      ],
      "execution_count": 182,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([3, 0, 0, 7])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 182
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1_v2MpufWUo3"
      },
      "source": [
        "### dim, keepdim"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xfQ43oFcVS43",
        "outputId": "d926fe8a-99cb-420c-da19-58a4e1afbb39"
      },
      "source": [
        "a"
      ],
      "execution_count": 183,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[-0.8184,  0.0473,  0.1040,  1.5820, -0.4505,  0.1631,  1.2550,  0.1290,\n",
              "         -1.5484, -0.7438],\n",
              "        [ 1.2569,  0.5309, -0.3137, -0.0231, -0.9398,  0.4437, -2.1725,  0.3998,\n",
              "          0.2672,  0.3625],\n",
              "        [ 0.8056, -0.4072, -0.3879,  0.4357,  0.3430, -0.0866, -0.5303, -0.7454,\n",
              "          0.0940, -0.6318],\n",
              "        [-0.5242,  0.4509, -0.4646,  0.7723, -0.3870, -1.1821, -0.0316,  0.8623,\n",
              "          0.2936, -0.0057]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 183
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vh7YMaHVXGyB",
        "outputId": "5b6d19b0-8bcd-42e5-946c-e9cf18bb2763"
      },
      "source": [
        "a.max(dim=1)"
      ],
      "execution_count": 188,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.return_types.max(values=tensor([1.5820, 1.2569, 0.8056, 0.8623]), indices=tensor([3, 0, 0, 7]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 188
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LSf5W9cAZEUr",
        "outputId": "705af8ec-7b79-47cc-ad4f-d48cef313d02"
      },
      "source": [
        "a.max(dim=1)[1]"
      ],
      "execution_count": 189,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([3, 0, 0, 7])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 189
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3277r1VqXKlO",
        "outputId": "46fce8a8-0442-423e-e0aa-4799d9ee67ea"
      },
      "source": [
        "a.argmax(dim=1)"
      ],
      "execution_count": 185,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([3, 0, 0, 7])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 185
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gOn5xgPdY8Xy",
        "outputId": "44291783-9e32-4985-c38b-6084279ee0ce"
      },
      "source": [
        "a.max(dim=1, keepdim=True)"
      ],
      "execution_count": 190,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.return_types.max(values=tensor([[1.5820],\n",
              "        [1.2569],\n",
              "        [0.8056],\n",
              "        [0.8623]]), indices=tensor([[3],\n",
              "        [0],\n",
              "        [0],\n",
              "        [7]]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 190
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ErxtyPhsZbp0",
        "outputId": "1898f277-ea94-4708-dc68-eab9bce510e1"
      },
      "source": [
        "a.argmax(dim=1, keepdim=True)"
      ],
      "execution_count": 191,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[3],\n",
              "        [0],\n",
              "        [0],\n",
              "        [7]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 191
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FffoDCkkc0VI"
      },
      "source": [
        "### top-k or k-th"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "845TlhzCefwa",
        "outputId": "d647d125-88c2-429e-c432-63d1a52eab2d"
      },
      "source": [
        "a"
      ],
      "execution_count": 199,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[-0.8184,  0.0473,  0.1040,  1.5820, -0.4505,  0.1631,  1.2550,  0.1290,\n",
              "         -1.5484, -0.7438],\n",
              "        [ 1.2569,  0.5309, -0.3137, -0.0231, -0.9398,  0.4437, -2.1725,  0.3998,\n",
              "          0.2672,  0.3625],\n",
              "        [ 0.8056, -0.4072, -0.3879,  0.4357,  0.3430, -0.0866, -0.5303, -0.7454,\n",
              "          0.0940, -0.6318],\n",
              "        [-0.5242,  0.4509, -0.4646,  0.7723, -0.3870, -1.1821, -0.0316,  0.8623,\n",
              "          0.2936, -0.0057]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 199
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iLXrI5_CZo3K",
        "outputId": "773d9022-0523-4a6f-86cb-ebb7322bf04d"
      },
      "source": [
        "a.topk(3, dim=1)"
      ],
      "execution_count": 196,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.return_types.topk(values=tensor([[1.5820, 1.2550, 0.1631],\n",
              "        [1.2569, 0.5309, 0.4437],\n",
              "        [0.8056, 0.4357, 0.3430],\n",
              "        [0.8623, 0.7723, 0.4509]]), indices=tensor([[3, 6, 5],\n",
              "        [0, 1, 5],\n",
              "        [0, 3, 4],\n",
              "        [7, 3, 1]]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 196
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G96NoeWZc7wC",
        "outputId": "4e2649d2-2e74-47f2-ab2e-a38e0c5cbece"
      },
      "source": [
        "a.topk(3, dim=1, largest=False)"
      ],
      "execution_count": 197,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.return_types.topk(values=tensor([[-1.5484, -0.8184, -0.7438],\n",
              "        [-2.1725, -0.9398, -0.3137],\n",
              "        [-0.7454, -0.6318, -0.5303],\n",
              "        [-1.1821, -0.5242, -0.4646]]), indices=tensor([[8, 0, 9],\n",
              "        [6, 4, 2],\n",
              "        [7, 9, 6],\n",
              "        [5, 0, 2]]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 197
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Xa0ldK7NeSGm",
        "outputId": "8c27f7ad-e135-48eb-db18-f063e61ae687"
      },
      "source": [
        "a.kthvalue(8, dim=1) # 第k小"
      ],
      "execution_count": 198,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.return_types.kthvalue(values=tensor([0.1631, 0.4437, 0.3430, 0.4509]), indices=tensor([5, 5, 4, 1]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 198
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OQWpb4LCeX89",
        "outputId": "0dc20c9b-c67a-4d8b-e0ee-49341596576e"
      },
      "source": [
        "a.kthvalue(3)"
      ],
      "execution_count": 200,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.return_types.kthvalue(values=tensor([-0.7438, -0.3137, -0.5303, -0.4646]), indices=tensor([9, 2, 6, 2]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 200
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TMfcfD5uejXt",
        "outputId": "c2a21910-206d-4aab-906e-bfbde799262b"
      },
      "source": [
        "a.kthvalue(3, dim=1)"
      ],
      "execution_count": 201,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.return_types.kthvalue(values=tensor([-0.7438, -0.3137, -0.5303, -0.4646]), indices=tensor([9, 2, 6, 2]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 201
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ha6x99tMfMky"
      },
      "source": [
        "### compare"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Qaz0855demT7",
        "outputId": "2fc9e961-427f-4d43-a89c-efd95fb69145"
      },
      "source": [
        "a>0"
      ],
      "execution_count": 202,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[False,  True,  True,  True, False,  True,  True,  True, False, False],\n",
              "        [ True,  True, False, False, False,  True, False,  True,  True,  True],\n",
              "        [ True, False, False,  True,  True, False, False, False,  True, False],\n",
              "        [False,  True, False,  True, False, False, False,  True,  True, False]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 202
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8WVwTNokfP65",
        "outputId": "b12c7c63-f778-4b99-d2d6-2d91cb8396da"
      },
      "source": [
        "torch.gt(a, 0)"
      ],
      "execution_count": 203,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[False,  True,  True,  True, False,  True,  True,  True, False, False],\n",
              "        [ True,  True, False, False, False,  True, False,  True,  True,  True],\n",
              "        [ True, False, False,  True,  True, False, False, False,  True, False],\n",
              "        [False,  True, False,  True, False, False, False,  True,  True, False]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 203
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ref6KmzJfVIh",
        "outputId": "1a72af11-5e6a-4c49-b76b-a14fe457e82d"
      },
      "source": [
        "a != 0"
      ],
      "execution_count": 204,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[True, True, True, True, True, True, True, True, True, True],\n",
              "        [True, True, True, True, True, True, True, True, True, True],\n",
              "        [True, True, True, True, True, True, True, True, True, True],\n",
              "        [True, True, True, True, True, True, True, True, True, True]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 204
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-uMGA0QifZLa",
        "outputId": "52e11d97-c0da-4e53-d79e-bab8092d0e8b"
      },
      "source": [
        "a = torch.ones(2, 3)\r\n",
        "b = torch.randn(2, 3)\r\n",
        "torch.eq(a, b)"
      ],
      "execution_count": 205,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[False, False, False],\n",
              "        [False, False, False]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 205
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2xifNbucfsxY",
        "outputId": "303e7e87-2d53-4559-948f-7c001205949c"
      },
      "source": [
        "torch.eq(a, a)"
      ],
      "execution_count": 206,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[True, True, True],\n",
              "        [True, True, True]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 206
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g3KykZO2fxwl",
        "outputId": "ecdafc80-e0e3-4d6b-a4ff-3455b40b0bb1"
      },
      "source": [
        "torch.equal(a, a)"
      ],
      "execution_count": 207,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 207
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eLLrfuPCgU-t"
      },
      "source": [
        "## 5.高阶操作\r\n",
        "\r\n",
        "* where\r\n",
        "* gather"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uODjvQWFpi5n"
      },
      "source": [
        "### where"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c1Bzu4FxgXX6",
        "outputId": "bde6bcc3-37ec-4254-b172-e990e72c38b8"
      },
      "source": [
        "cond = torch.tensor([[0.6, 0.7], \r\n",
        "        [0.8, 0.4]]).double()\r\n",
        "cond"
      ],
      "execution_count": 222,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.6000, 0.7000],\n",
              "        [0.8000, 0.4000]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 222
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "F19WKXfSiFKo",
        "outputId": "6e0ada75-9db9-45be-8ff0-8b3322e025f2"
      },
      "source": [
        "import numpy as np\r\n",
        "\r\n",
        "a = torch.tensor([[0, 0],\r\n",
        "     [0, 0]]).double()\r\n",
        "b = torch.tensor([[1, 1],\r\n",
        "     [1, 1]]).double()\r\n",
        "a.type()\r\n"
      ],
      "execution_count": 223,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'torch.DoubleTensor'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 223
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vIloAikJiTLR",
        "outputId": "77a38c99-6206-47f2-9005-7b0d5e099665"
      },
      "source": [
        "torch.where(cond>0.5, a, b)"
      ],
      "execution_count": 225,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0., 0.],\n",
              "        [0., 1.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 225
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NasXx3GfkYyv"
      },
      "source": [
        "### gather\r\n",
        "查表\r\n",
        "\r\n",
        "torch.gather(input, dim, index, out=None) -> Tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X6DBWXBEiXr6",
        "outputId": "87a409c3-64d4-4a11-8962-0b2132da2d2e"
      },
      "source": [
        "prob = torch.randn(4, 10)\r\n",
        "prob"
      ],
      "execution_count": 241,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ 0.8064,  1.7472,  2.3267,  0.2556,  0.7068, -1.3559, -0.7067, -0.9383,\n",
              "          0.6252,  1.0672],\n",
              "        [-0.9990,  2.9914, -0.3745, -0.7212, -0.9276, -2.1059, -0.5704, -0.7625,\n",
              "         -0.4164, -1.2965],\n",
              "        [-0.3657,  2.7413, -0.0811,  0.8845, -0.3860,  0.4786, -1.9337, -0.9188,\n",
              "         -0.5971, -1.0415],\n",
              "        [-0.6080,  1.0508, -1.1711, -0.6335, -0.4743, -0.5340,  1.1351,  1.3671,\n",
              "         -1.0961, -0.7392]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 241
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GJyuxreVn-Sd",
        "outputId": "f7a8ca36-0bee-4707-95e7-a099c908dd98"
      },
      "source": [
        "idx = prob.topk(dim=1, k=3)\r\n",
        "idx"
      ],
      "execution_count": 242,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.return_types.topk(values=tensor([[ 2.3267,  1.7472,  1.0672],\n",
              "        [ 2.9914, -0.3745, -0.4164],\n",
              "        [ 2.7413,  0.8845,  0.4786],\n",
              "        [ 1.3671,  1.1351,  1.0508]]), indices=tensor([[2, 1, 9],\n",
              "        [1, 2, 8],\n",
              "        [1, 3, 5],\n",
              "        [7, 6, 1]]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 242
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mNx7s7JMoD7q",
        "outputId": "cb18290e-a7a8-4dca-8cdf-deb4f19fded7"
      },
      "source": [
        "idx = idx[1]\r\n",
        "idx"
      ],
      "execution_count": 243,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[2, 1, 9],\n",
              "        [1, 2, 8],\n",
              "        [1, 3, 5],\n",
              "        [7, 6, 1]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 243
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hto48M_0oI96",
        "outputId": "4a9df360-3006-4456-f033-29003d4264d4"
      },
      "source": [
        "label = torch.arange(10) + 100\r\n",
        "label"
      ],
      "execution_count": 244,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([100, 101, 102, 103, 104, 105, 106, 107, 108, 109])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 244
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lb8Uvs48oRsh",
        "outputId": "8fdc422e-67e0-44ea-eeca-5bda29618b81"
      },
      "source": [
        "torch.gather(label.expand(4, 10), dim=1, index=idx.long())"
      ],
      "execution_count": 245,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[102, 101, 109],\n",
              "        [101, 102, 108],\n",
              "        [101, 103, 105],\n",
              "        [107, 106, 101]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 245
        }
      ]
    }
  ]
}